Bayesian Epistemology
   HOME

TheInfoList



OR:

Bayesian epistemology is a formal approach to various topics in
epistemology Epistemology (; ), or the theory of knowledge, is the branch of philosophy concerned with knowledge. Epistemology is considered a major subfield of philosophy, along with other major subfields such as ethics, logic, and metaphysics. Episte ...
that has its roots in
Thomas Bayes Thomas Bayes ( ; 1701 7 April 1761) was an English statistician, philosopher and Presbyterian minister who is known for formulating a specific case of the theorem that bears his name: Bayes' theorem. Bayes never published what would become his ...
' work in the field of probability theory. One advantage of its formal method in contrast to traditional epistemology is that its concepts and theorems can be defined with a high degree of precision. It is based on the idea that beliefs can be interpreted as
subjective probabilities Bayesian probability is an interpretation of the concept of probability, in which, instead of frequency or propensity of some phenomenon, probability is interpreted as reasonable expectation representing a state of knowledge or as quantification o ...
. As such, they are subject to the laws of
probability theory Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expressing it through a set o ...
, which act as the norms of
rationality Rationality is the quality of being guided by or based on reasons. In this regard, a person acts rationally if they have a good reason for what they do or a belief is rational if it is based on strong evidence. This quality can apply to an abil ...
. These norms can be divided into static constraints, governing the rationality of beliefs at any moment, and dynamic constraints, governing how rational agents should change their beliefs upon receiving new evidence. The most characteristic Bayesian expression of these principles is found in the form of
Dutch book In gambling, a Dutch book or lock is a set of odds and bets, established by the bookmaker, that ensures that the bookmaker will profit—at the expense of the gamblers—regardless of the outcome of the event (a horse race, for example) on which ...
s, which illustrate irrationality in agents through a series of bets that lead to a loss for the agent no matter which of the probabilistic events occurs. Bayesians have applied these fundamental principles to various epistemological topics but Bayesianism does not cover all topics of traditional epistemology. The problem of confirmation in the
philosophy of science Philosophy of science is a branch of philosophy concerned with the foundations, methods, and implications of science. The central questions of this study concern what qualifies as science, the reliability of scientific theories, and the ultim ...
, for example, can be approached through the Bayesian ''principle of conditionalization'' by holding that a piece of evidence confirms a theory if it raises the likelihood that this theory is true. Various proposals have been made to define the concept of
coherence Coherence, coherency, or coherent may refer to the following: Physics * Coherence (physics), an ideal property of waves that enables stationary (i.e. temporally and spatially constant) interference * Coherence (units of measurement), a deriv ...
in terms of probability, usually in the sense that two propositions cohere if the probability of their conjunction is higher than if they were neutrally related to each other. The Bayesian approach has also been fruitful in the field of
social epistemology Social epistemology refers to a broad set of approaches that can be taken in epistemology (the study of knowledge) that construes human knowledge as a collective achievement. Another way of characterizing social epistemology is as the evaluation o ...
, for example, concerning the problem of
testimony In law and in religion, testimony is a solemn attestation as to the truth of a matter. Etymology The words "testimony" and "testify" both derive from the Latin word ''testis'', referring to the notion of a disinterested third-party witness. La ...
or the problem of group belief. Bayesianism still faces various theoretical objections that have not been fully solved.


Relation to traditional epistemology

Traditional epistemology and Bayesian epistemology are both forms of epistemology, but they differ in various respects, for example, concerning their methodology, their interpretation of belief, the role justification or confirmation plays in them and some of their research interests. Traditional epistemology focuses on topics such as the analysis of the ''nature of knowledge'', usually in terms of
justified true belief Definitions of knowledge try to determine the essential features of knowledge. Closely related terms are conception of knowledge, theory of knowledge, and analysis of knowledge. Some general features of knowledge are widely accepted among philoso ...
s, the ''sources of knowledge'', like perception or testimony, the ''structure of a body of knowledge'', for example in the form of
foundationalism Foundationalism concerns philosophical theories of knowledge resting upon non-inferential justified belief, or some secure foundation of certainty such as a conclusion inferred from a basis of sound premises.Simon Blackburn, ''The Oxford Dictio ...
or
coherentism In philosophical epistemology, there are two types of coherentism: the coherence theory of truth; and the coherence theory of justification (also known as epistemic coherentism). Coherent truth is divided between an anthropological approach, whic ...
, and the problem of ''philosophical skepticism'' or the question of whether knowledge is possible at all. These inquiries are usually based on epistemic intuitions and regard beliefs as either present or absent. Bayesian epistemology, on the other hand, works by formalizing concepts and problems, which are often vague in the traditional approach. It thereby focuses more on mathematical intuitions and promises a higher degree of precision. It sees belief as a continuous phenomenon that comes in various degrees, so-called ''credences''. Some Bayesians have even suggested that the regular notion of belief should be abandoned. But there are also proposals to connect the two, for example, the ''Lockean thesis'', which defines belief as credence above a certain threshold. Justification plays a central role in traditional epistemology while Bayesians have focused on the related notions of confirmation and disconfirmation through evidence. The notion of evidence is important for both approaches but only the traditional approach has been interested in studying the sources of evidence, like perception and memory. Bayesianism, on the other hand, has focused on the role of evidence for rationality: how someone's credence should be adjusted upon receiving new evidence. There is an analogy between the Bayesian norms of rationality in terms of probabilistic laws and the traditional norms of rationality in terms of deductive consistency. Certain traditional problems, like the topic of skepticism about our knowledge of the external world, are difficult to express in Bayesian terms.


Fundamentals

Bayesian epistemology is based only on a few fundamental principles, which can be used to define various other notions and can be applied to many topics in epistemology. At their core, these principles constitute constraints on how we should assign credences to propositions. They determine what an ideally rational agent would believe. The basic principles can be divided into synchronic or static principles, which govern how credences are to be assigned at any moment, and diachronic or dynamic principles, which determine how the agent should change her beliefs upon receiving new evidence. The ''axioms of probability'' and the ''principal principle'' belong to the static principles while the principle of conditionalization governs the dynamic aspects as a form of probabilistic inference. The most characteristic Bayesian expression of these principles is found in the form of
Dutch book In gambling, a Dutch book or lock is a set of odds and bets, established by the bookmaker, that ensures that the bookmaker will profit—at the expense of the gamblers—regardless of the outcome of the event (a horse race, for example) on which ...
s, which illustrate irrationality in agents through a series of bets that lead to a loss for the agent no matter which of the probabilistic events occurs. This test for determining irrationality has been referred to as the "pragmatic self-defeat test".


Beliefs, probability and bets

One important difference to traditional epistemology is that Bayesian epistemology focuses not on the notion of simple belief but on the notion of degrees of belief, so-called ''credences''. This approach tries to capture the idea of certainty: we believe in all kinds of claims but we are more certain about some, like that the earth is round, than about others, like that
Plato Plato ( ; grc-gre, Πλάτων ; 428/427 or 424/423 – 348/347 BC) was a Greek philosopher born in Athens during the Classical period in Ancient Greece. He founded the Platonist school of thought and the Academy, the first institution ...
was the author of the
First Alcibiades The ''First Alcibiades'', also referred to as ''Alcibiades Major'' and abbreviated as ''Alcibiades I'' ( el, Ἀλκιβιάδης αʹ), is a dialogue depicting Socrates in conversation with Alcibiades. It is ascribed to Plato, although scholars ...
. These degrees come in values between 0 and 1. 0 corresponds to full disbelief, 1 corresponds to full belief and 0.5 corresponds to suspension of belief. According to the Bayesian interpretation of probability, credences stand for subjective probabilities. Following
Frank P. Ramsey Frank Plumpton Ramsey (; 22 February 1903 – 19 January 1930) was a British people, British philosopher, mathematician, and economist who made major contributions to all three fields before his death at the age of 26. He was a close friend of L ...
, they are interpreted in terms of the willingness to bet money on a claim. So having a credence of 0.8 (i.e. 80 %) that your favorite soccer team will win the next game would mean being willing to bet up to four dollars for the chance to make one dollar profit. This account draws a tight connection between Bayesian epistemology and
decision theory Decision theory (or the theory of choice; not to be confused with choice theory) is a branch of applied probability theory concerned with the theory of making decisions based on assigning probabilities to various factors and assigning numerical ...
. It might seem that betting-behavior is only one special area and as such not suited for defining such a general notion as credences. But, as Ramsey argues, we bet all the time when understood in the widest sense. For example, in going to the train station, we bet on the train being there on time, otherwise we would have stayed at home. It follows from the interpretation of credence in terms of willingness to make bets that it would be irrational to ascribe a credence of 0 or 1 to any proposition, except for
contradictions In traditional logic, a contradiction occurs when a proposition conflicts either with itself or established fact. It is often used as a tool to detect disingenuous beliefs and bias. Illustrating a general tendency in applied logic, Aristotle's ...
and tautologies. The reason for this is that ascribing these extreme values would mean that one would be willing to bet anything, including one's life, even if the payoff was minimal. Another negative side-effect of such extreme credences is that they are permanently fixed and cannot be updated anymore upon acquiring new evidence. This central tenet of Bayesianism, that credences are interpreted as subjective probabilities and are therefore governed by the norms of probability, has been referred to as ''probabilism''. These norms express the nature of the credences of ideally rational agents. They do not put demands on what credence we should have on any single given belief, for example, whether it will rain tomorrow. Instead, they constrain the system of beliefs as a whole. For example, if your credence that it will rain tomorrow is 0.8 then your credence in the opposite proposition, i.e. that it will not rain tomorrow, should be 0.2, not 0.1 or 0.5. According to Stephan Hartmann and Jan Sprenger, the axioms of probability can be expressed through the following two laws: (1) P(A) = 1 for any tautology A; (2) For incompatible (mutually exclusive) propositions A and B, P(A \lor B) = P(A) + P(B). Another important Bayesian principle of degrees of beliefs is the ''principal principle'' due to David Lewis. It states that our knowledge of objective probabilities should correspond to our subjective probabilities in the form of credences. So if you know that the objective chance of a coin landing heads is 50% then your credence that the coin will land heads should be 0.5. The ''axioms of probability'' together with the ''principal principle'' determines the ''static'' or ''synchronic'' aspect of rationality: what an agent's beliefs should be like when only considering one moment. But rationality also involves a ''dynamic'' or ''diachronic'' aspect, which comes to play for changing one's credences upon being confronted with new evidence. This aspect is determined by the ''principle of conditionalization''.


Principle of conditionalization

The ''principle of conditionalization'' governs how the agent's credence in a hypothesis should change upon receiving new evidence for or against this hypothesis. As such, it expresses the dynamic aspect of how ideal rational agents would behave. It is based on the notion of
conditional probability In probability theory, conditional probability is a measure of the probability of an event occurring, given that another event (by assumption, presumption, assertion or evidence) has already occurred. This particular method relies on event B occur ...
, which is the measure of the probability that one event occurs given that another event has already occurred. The unconditional probability that A will occur is usually expressed as P(A) while the conditional probability that A will occur given that B has already occurred is written as P(A \mid B). For example, the probability of flipping a coin two times and the coin landing heads two times is only 25%. But the conditional probability of this occurring given that the coin has landed heads on the first flip is then 50%. The ''principle of conditionalization'' applies this idea to credences: we should change our credence that the coin will land heads two times upon receiving evidence that it has already landed heads on the first flip. The probability assigned to the hypothesis before the event is called
prior probability In Bayesian statistical inference, a prior probability distribution, often simply called the prior, of an uncertain quantity is the probability distribution that would express one's beliefs about this quantity before some evidence is taken into ...
. The probability afterward is called
posterior probability The posterior probability is a type of conditional probability that results from updating the prior probability with information summarized by the likelihood via an application of Bayes' rule. From an epistemological perspective, the posterior p ...
. According to the ''simple principle of conditionalization'', this can be expressed in the following way: P_\text(H) = P_\text(H \mid E) = \frac . So the posterior probability that the hypothesis is true is equal to the conditional prior probability that the hypothesis is true relative to the evidence, which is equal to the prior probability that both the hypothesis and the evidence are true, divided by the prior probability that the evidence is true. The original expression of this principle, referred to as
Bayes' theorem In probability theory and statistics, Bayes' theorem (alternatively Bayes' law or Bayes' rule), named after Thomas Bayes, describes the probability of an event, based on prior knowledge of conditions that might be related to the event. For examp ...
, can be directly deduced from this formulation. The ''simple principle of conditionalization'' makes the assumption that our credence in the acquired evidence, i.e. its posterior probability, is 1, which is unrealistic. For example, scientists sometimes need to discard previously accepted evidence upon making new discoveries, which would be impossible if the corresponding credence was 1. An alternative form of conditionalization, proposed by
Richard Jeffrey Richard Carl Jeffrey (August 5, 1926 – November 9, 2002) was an American philosopher, logician, and probability theorist. He is best known for developing and championing the philosophy of radical probabilism and the associated heuristic of pr ...
, adjusts the formula to take the probability of the evidence into account: P_\text(H) = P_\text(H \mid E) \cdot P_\text(E) + P_\text(H \mid \lnot E) \cdot P_\text(\lnot E).


Dutch books

A
Dutch book In gambling, a Dutch book or lock is a set of odds and bets, established by the bookmaker, that ensures that the bookmaker will profit—at the expense of the gamblers—regardless of the outcome of the event (a horse race, for example) on which ...
is a series of bets that necessarily results in a loss. An agent is vulnerable to a Dutch book if her credences violate the laws of probability. This can be either in synchronic cases, in which the conflict happens between beliefs held at the same time, or in diachronic cases, in which the agent does not respond properly to new evidence. In the most simple synchronic case, only two credences are involved: the credence in a proposition and in its negation. The laws of probability hold that these two credences together should amount to 1 since either the proposition or its negation are true. Agents who violate this law are vulnerable to a synchronic Dutch book. For example, given the proposition that it will rain tomorrow, suppose that an agent's degree of belief that it is true is 0.51 and the degree that it is false is also 0.51. In this case, the agent would be willing to accept two bets at $0.51 for the chance to win $1: one that it will rain and another that it will not rain. The two bets together cost $1.02, resulting in a loss of $0.02, no matter whether it will rain or not. The principle behind diachronic Dutch books is the same, but they are more complicated since they involve making bets before and after receiving new evidence and have to take into account that there is a loss in each case no matter how the evidence turns out to be. There are different interpretations about what it means that an agent is vulnerable to a Dutch book. On the traditional interpretation, such a vulnerability reveals that the agent is irrational since she would willingly engage in behavior that is not in her best self-interest. One problem with this interpretation is that it assumes logical omniscience as a requirement for rationality, which is problematic especially in complicated diachronic cases. An alternative interpretation uses Dutch books as "a kind of heuristic for determining when one's degrees of belief have the potential to be pragmatically self-defeating". This interpretation is compatible with holding a more realistic view of rationality in the face of human limitations. Dutch books are closely related to the ''axioms of probability''. The ''Dutch book theorem'' holds that only credence assignments that do not follow the axioms of probability are vulnerable to Dutch books. The ''converse Dutch book theorem'' states that no credence assignment following these axioms is vulnerable to a Dutch book.


Applications


Confirmation theory

In the
philosophy of science Philosophy of science is a branch of philosophy concerned with the foundations, methods, and implications of science. The central questions of this study concern what qualifies as science, the reliability of scientific theories, and the ultim ...
, ''confirmation'' refers to the relation between a piece of evidence and a hypothesis ''confirmed'' by it. ''Confirmation theory'' is the study of confirmation and disconfirmation: how scientific hypotheses are supported or refuted by evidence. ''Bayesian confirmation theory'' provides a model of confirmation based on the ''principle of conditionalization''. A piece of evidence confirms a theory if the conditional probability of that theory relative to the evidence is higher than the unconditional probability of the theory by itself. Expressed formally: P(H \mid E) > P(H). If the evidence lowers the probability of the hypothesis then it disconfirms it. Scientists are usually not just interested in whether a piece of evidence supports a theory but also in how much support it provides. There are different ways how this degree can be determined. The simplest version just measures the difference between the conditional probability of the hypothesis relative to the evidence and the unconditional probability of the hypothesis, i.e. the degree of support is P(H \mid E) - P(H). The problem with measuring this degree is that it depends on how certain the theory already is prior to receiving the evidence. So if a scientist is already very certain that a theory is true then one further piece of evidence will not affect her credence much, even if the evidence would be very strong. There are other constraints for how an evidence measure should behave, for example, surprising evidence, i.e. evidence that had a low probability on its own, should provide more support. Scientists are often faced with the problem of having to decide between two competing theories. In such cases, the interest is not so much in absolute confirmation, or how much a new piece of evidence would support this or that theory, but in relative confirmation, i.e. in which theory is supported more by the new evidence. A well-known problem in confirmation theory is
Carl Gustav Hempel Carl Gustav "Peter" Hempel (January 8, 1905 – November 9, 1997) was a German writer, philosopher, logician, and epistemologist. He was a major figure in logical empiricism, a 20th-century movement in the philosophy of science. He is espec ...
's
raven paradox The raven paradox, also known as Hempel's paradox, Hempel's ravens, or rarely the paradox of indoor ornithology, is a paradox arising from the question of what constitutes evidence for the truth of a statement. Observing objects that are neither ...
. Hempel starts by pointing out that seeing a black raven counts as evidence for the hypothesis that ''all ravens are black'' while ''seeing a green apple'' is usually not taken to be evidence for or against this hypothesis. The paradox consists in the consideration that the hypothesis "all ravens are black" is logically equivalent to the hypothesis "if something is not black, then it is not a raven". So since seeing a green apple counts as evidence for the second hypothesis, it should also count as evidence for the first one. Bayesianism allows that seeing a green apple supports the raven-hypothesis while explaining our initial intuition otherwise. This result is reached if we assume that seeing a green apple provides minimal but still positive support for the raven-hypothesis while spotting a black raven provides significantly more support.


Coherence

''Coherence'' plays a central role in various epistemological theories, for example, in the
coherence theory of truth Coherence theories of truth characterize truth as a property of whole systems of propositions that can be ascribed to individual propositions only derivatively according to their coherence with the whole. While modern coherence theorists hold that ...
or in the
coherence theory of justification In philosophical epistemology, there are two types of coherentism: the coherence theory of truth; and the coherence theory of justification (also known as epistemic coherentism). Coherent truth is divided between an anthropological approach, whic ...
. It is often assumed that sets of beliefs are more likely to be true if they are coherent than otherwise. For example, we would be more likely to trust a detective who can connect all the pieces of evidence into a coherent story. But there is no general agreement as to how coherence is to be defined. Bayesianism has been applied to this field by suggesting precise definitions of coherence in terms of probability, which can then be employed to tackle other problems surrounding coherence. One such definition was proposed by Tomoji Shogenji, who suggests that the coherence between two beliefs is equal to the probability of their conjunction divided by the probabilities of each by itself, i.e. Coherence(A, B) = \frac. Intuitively, this measures how likely it is that the two beliefs are true at the same time, compared to how likely this would be if they were neutrally related to each other. The coherence is high if the two beliefs are relevant to each other. Coherence defined this way is relative to a credence assignment. This means that two propositions may have high coherence for one agent and a low coherence for another agent due to the difference in prior probabilities of the agents' credences.


Social epistemology

''
Social epistemology Social epistemology refers to a broad set of approaches that can be taken in epistemology (the study of knowledge) that construes human knowledge as a collective achievement. Another way of characterizing social epistemology is as the evaluation o ...
'' studies the relevance of social factors for knowledge. In the field of science, for example, this is relevant since individual scientists have to place their trust in some claimed discoveries of other scientists in order to progress. The Bayesian approach can be applied to various topics in social epistemology. For example, probabilistic reasoning can be used in the field of
testimony In law and in religion, testimony is a solemn attestation as to the truth of a matter. Etymology The words "testimony" and "testify" both derive from the Latin word ''testis'', referring to the notion of a disinterested third-party witness. La ...
to evaluate how reliable a given report is. In this way, it can be formally shown that witness reports that are probabilistically independent of each other provide more support than otherwise. Another topic in social epistemology concerns the question of how to aggregate the beliefs of the individuals within a group to arrive at the belief of the group as a whole. Bayesianism approaches this problem by aggregating the probability assignments of the different individuals.


Objections


Problem of priors

In order to draw probabilistic inferences based on new evidence, it is necessary to already have a prior probability assigned to the proposition in question. But this is not always the case: there are many propositions that the agent never considered and therefore lacks a credence for. This problem is usually solved by assigning a probability to the proposition in question in order to learn from the new evidence through conditionalization. The ''problem of priors'' concerns the question of how this initial assignment should be done. ''Subjective Bayesians'' hold that there are no or few constraints besides probabilistic coherence that determine how we assign the initial probabilities. The argument for this freedom in choosing the initial credence is that the credences will change as we acquire more evidence and will converge on the same value after enough steps no matter where we start. ''Objective Bayesians'', on the other hand, assert that there are various constraints that determine the initial assignment. One important constraint is the
principle of indifference The principle of indifference (also called principle of insufficient reason) is a rule for assigning epistemic probabilities. The principle of indifference states that in the absence of any relevant evidence, agents should distribute their cre ...
. It states that the credences should be distributed equally among all the possible outcomes. For example, the agent wants to predict the color of balls drawn from an urn containing only red and black balls without any information about the ratio of red to black balls. Applied to this situation, the principle of indifference states that the agent should initially assume that the probability to draw a red ball is 50%. This is due to symmetric considerations: it is the only assignment in which the prior probabilities are invariant to a change in label. While this approach works for some cases it produces paradoxes in others. Another objection is that one should not assign prior probabilities based on initial ignorance.


Problem of logical omniscience

The norms of rationality according to the standard definitions of Bayesian epistemology assume ''logical omniscience'': the agent has to make sure to exactly follow all the laws of probability for all her credences in order to count as rational. Whoever fails to do so is vulnerable to Dutch books and is therefore irrational. This is an unrealistic standard for human beings, as critics have pointed out.


Problem of old evidence

The ''problem of old evidence'' concerns cases in which the agent does not know at the time of acquiring a piece of evidence that it confirms a hypothesis but only learns about this supporting-relation later. Normally, the agent would increase her belief in the hypothesis after discovering this relation. But this is not allowed in ''Bayesian confirmation theory'' since conditionalization can only happen upon a change of the probability of the evidential statement, which is not the case. For example, the observation of certain anomalies in the orbit of Mercury is evidence for the
theory of general relativity General relativity, also known as the general theory of relativity and Einstein's theory of gravity, is the geometric theory of gravitation published by Albert Einstein in 1915 and is the current description of gravitation in modern physics. G ...
. But this data had been obtained before the theory was formulated, thereby counting as old evidence.


See also

*
Bayesian statistics Bayesian statistics is a theory in the field of statistics based on the Bayesian interpretation of probability where probability expresses a ''degree of belief'' in an event. The degree of belief may be based on prior knowledge about the event, ...
*
Bayesian probability Bayesian probability is an Probability interpretations, interpretation of the concept of probability, in which, instead of frequentist probability, frequency or propensity probability, propensity of some phenomenon, probability is interpreted as re ...
*
Bayesian inference Bayesian inference is a method of statistical inference in which Bayes' theorem is used to update the probability for a hypothesis as more evidence or information becomes available. Bayesian inference is an important technique in statistics, a ...
*
Probability interpretations The word probability has been used in a variety of ways since it was first applied to the mathematical study of games of chance. Does probability measure the real, physical, tendency of something to occur, or is it a measure of how strongly one be ...


References


Further reading

* {{cite book , first=Michael G. , last=Titelbaum , title=Fundamentals of Bayesian Epistemology , location= , publisher=Oxford University Press , year=2022 , isbn=978-0-19-870761-5 Bayesian inference Bayesian statistics Coherentism Epistemological theories Epistemology Philosophy of mathematics Philosophy of science Probability interpretations